None Notebook

This notebook contains material from cbe61622; content is available on Github.

< 10.11 Computer Vision Case Study: Finding Particles in Images | Contents | 10.13 Computer Vision Case Study: Creating an Application >

Open in Colab

Download

10.12 Computer Vision Case Study: Creating an Application

10.12.1 What did we learn about our application?

10.12.1 Test Driven Development, or Festina Lente ("make haste, slowly")

Our goal is to create a Python module for particle labeling and counting. We anticipate testing the approach using representative images taken under different conditions, then embedding the code into a user application, perhaps with a GUI interface, running on a microcontroller with integrated camera.

For these purposes, we will create several Python classes to streamline the workflow. It's important for the workflow to be flexible and adaptable to accomodate experimentation.

Test Driven Development

We'll simultaneously writing test code to verify that our classes correctly implement steps in the workflow. Test driven development is a techique where units tests are written to guide code development. Test driven development is normally done in Python IDEs with formal software tools for unit testing, but the core principals can also be applied to smaller research projects created in Jupyter notebooks.

The process for code development organized as follows:

Unit Test

Unit tests are code that tests the function the smallest piece of code that can be logically isolated in a larger system. There are software tools specifically designed to support unit testing of Python modules. We can also create simple tests using Python's builtin assert statement.

Remember:

  1. Untested code is broken code.
  2. Don't waste your time writing untested code.

Simplicity Critera

10.12.1.1 Labeler Class

The Labeler class will be the primary interface to the application. When working from the Python console, the user use Labeler class to create labelers that locate and label particles in one or more images. Typical usage would be

labeler = Labeler(parameter1, parameter2, ...)

where parameters specify the workflow that will be used. Once created, a typical use of labeler would be might be as follows:

labeler.read(filepath)
labeler.show(ax)

Note that we pass an plotting axis to the .show() class method. This allows the application to control plotting and layouts, and simplifies the Labeler class.

Let's start by creating a cell that would test these features.

Exercise

Before going further, attempt to write some Python code that successfully passes the tests in the cell above.

10.12.1.2 Loading image files

We will read files in RGB format and convert to numpy ndarray. We'll create tests that verify the images are correctly load, and then write code until the tests succeed. We will save the filepath to label plots, and save the image data as a NumPy array in RGB format.

Note that in this cell we cheat a bit on TDD, doing a bit of iteration between the tests and application code. The goal is to end up the desired features and test code that completely covers the application code.

10.12.1.3 Image Rescaling and Cropping

The test images show the images appear with different sizes, and extraneous information at the boundaries. In this cell we add functions to rescale and crop the image. The display is changed to more clearly label the coordinate system. The Python native slice() function is used to specify the crop.

10.12.1.4 Channel Class

The next steps of the workflow involve the creation of new images through affine combination of the image channels and various filtering and morphological transformations. For this purpose we create a Channel class to represent these derivative images. We will assume

The next step in the workflow is to separate the image into channels. We will implement a channel as a two dimensional NumPy array with same height and width as the rescaled and cropped image. The data entries are eight bit unsigned integers One extra consideration is that we need additional channels to store intermediate results.

In the next cell we'll create a Channel class. Later we will integrate Channel class into the main Labeler application.

What we seek is a Channel class where

channel = Channel()              # create Channel object with empty channel data
channel = Channel(img_2Ddata)    # create Channel object from 2D numpy channel data
channel.histogram                # return histogram
channel.show(ax)                 # show the channel on a specified axis
channel.show_histogram(ax)       # show the histogram on a specified axis

It will sometimes be helpful to display a channel with additional color information, particularly with regard to showing multiple histograms on a single axis. For this reason we also desire the following features:

channel.show(ax, color)            # show the channel on a specified axis
channel.show_histogram(ax, color)  # show the histogram on a specified axis

where color is one of "k", "r", "g", or "b".

We will need to access the image data from a channel. To facilitate access, and to hide details of how the image data is stored inside the class, we let

channel()                          # return image data as a NumPy array

The testing goal is to provide complete coverage of the Channel class.

10.12.1.5 Integrating Channels into Labeler

Next we use the Channels object inside Labeler. The functionality we seek is to create channels from the core image. A channel is create upon first reference. For example, to create a gray scalel approximation to the reference image.

r, g, b = labeler.img
labeler.channels['k']((r + g + b)/3)

Data from the channel is accessed by a function call to the instance.

bw = labeler.channels['k']()

10.12.1.6 Additional Channel Methods

Most of the processing work will be done with Channels. Following the experimental explorations, at a minimum we know we'll need

We cut and past the prior Channel class, create tests, then code the corresponding methods.

10.12.2 Particle Class

10.12.3 Classes

How can we break this application down into classes?

What kinds of coding objects do work with?

Main application class: Labeler

Utility classes

Particle Labeling Classes

To facilitate embedded use in a device, the next step is to consolidate these procedures into a class.

10.12.4 Demonstrations

< 10.11 Computer Vision Case Study: Finding Particles in Images | Contents | 10.13 Computer Vision Case Study: Creating an Application >

Open in Colab

Download